Wastewater Proficiency Testing (PT WWS4) Report

Summary of Key Findings

• A total of 22 laboratories participated in this exercise
• Assay performance was clearly genotype-dependent; however, interpretation should be limited to the specific measles spike materials used for this exercise
  • The Paulos et al. assay (WWSCAN) showed the highest sensitivity for the D8 spike, but failed to detect the B3 spike (Figure 1)
• False detections (cross-reactivity) were rare, though some misclassifications of the B3 genotype and vaccine-derived signal did occur (most commonly with the Wu et al. assay implemented with a 55 degree C annealing temperature, Figure 3)
• Whole genome sequencing data were critical to understanding assay performance Section 2.2
• Unexpected assay behavior was observed during internal testing, with likely artifactual amplification observed with a clinical assay, specific to the D8 genotype (Section 4.2)

Cross-Assay Comparisons

Participating laboratories tested PT samples using the following assays:

• Hummel et al. (2006): A clinical assay that does not discriminate between wild-type and vaccine-derived signal. No participating public health laboratories are using this assay for their routine measles surveillance; the datasets represented here come from WSLH internal testing and a participating commercial laboratory.
• Wu et al. (2024): A widely-used assay developed by the Stadler Lab at Rice University. Implementation of this assay varies widely by laboratory, both in what probes are used and the thermal cycling conditions used. The conditions used for internal testing are aligned with recently-released CDC recommendations (62 degree annealing temp and WT1 probe alone).
• Paulos et al. (2025): A modified version of an assay published in Roy et al. (2017), employed by WastewaterSCAN as well as other programs for routine surveillance.
• GT Molecular: Target sequences for commercial assays #100835 and #100831 are not publicly disclosed. GT kit #101035 represents a commercial implementation of a recently CDC-recommended assay for wild-type measles and corresponds to the Wu et al. assay, employing the WT1 probe only. Results from catalogue #101035 are therefore grouped with Wu et al. for the purposes of this analysis.

Assay Name	# of Results
Wu et al. (Rice)	10
GT Molecular	7
Paulos et al. (WWSCAN)	6
Hummel et al.	2

Table 1: Number of datasets represented in comparisons below, categorized by assay used. Note that two laboratories (including WSLH) submitted results from multiple assays.

Assay Regions

The diagram below illustrates the regions of the measles virus targeted by assays used by participating laboratories, as well as the N450 genotyping region.

Sequence Information

The following plots display the primer and probe sequences of the PCR assays used by participating laboratories alongside reference material sequences for the corresponding assay regions. Positions displayed in red text indicate a mismatch between primer or probe sequences and sequenced reference material; black text indicates the presence of a degenerate base in the primer sequence.

Wu et al. Assay (Rice University, Stadler Lab)

Assay information available at https://doi.org/10.1021/acs.est.4c05344. Reference material sequenced courtesy of Colorado NWSS Center of Excellence.

Paulos et al. Assay (WWSCAN, modified Roy et al.)

Assay information available at https://doi.org/10.1101/2025.07.18.25331801. Reference material sequenced courtesy of Colorado NWSS Center of Excellence.

Hummel et al. Assay (CDC clinical assay)

Assay information available at https://doi.org/10.1016/j.jviromet.2005.10.006. Reference material sequenced courtesy of Colorado NWSS Center of Excellence. Note that this assay is not designed to distinguish between WT and VX signal.

Assay x Genotype Effects

Interpretation of assay performance across genotypes is limited to the spiked reference materials used in this PT only.

Figure 1: Quantification of wild-type measles genotypes B3 and D8 in buffer and wastewater, respectively. All participating laboratories represented.

The assays used showed quite different sensitivity to the B3 and D8 genotypes. The Paulos et al. assay provided the highest quantification for the D8 spike, while the Hummel et al. assay exhibited the highest sensitivity for the B3 spike. The Paulos et al. assay failed to detect the B3 spike used in this exercise. This is likely attributable to the single base mismatch to the B3 template with Paulos et al. probe (note that this wild-type specific assay also harbors only a single mismatch to the vaccine genotype used here). However, we should stress that internal validation of the Paulos et al. assay was successfully completed using clinical RNA of the B3 genotype (from an alternative source, with sequence unknown). This further highlights the importance of sequencing data for interpreting assay performance.

Figure 2: Quantification of vaccine-derived measles. A “MeV WT” result (left column) indicates erroneous classification of vaccine-derived signal as wild-type, keeping in mind that the Hummel et al. assay is not designed to discriminate WT from vaccine-derived signal. The only assay currently employed by participating public health laboratories designed to identify vaccine-derived signal is the Wu et al. assay with the vaccine (VX) probe.

No participating laboratories reported the presence of wild-type measles (above LOD) in the ‘WW Only’ background sample, and only one laboratory reported a detection of wild-type signal in samples containing only the vaccine genotype. Of the assays represented, erroneous trace detections were the most common in the Paulos et al. assay.

Assay Condition Effects

Implementations of the Wu et al. assay as well as the Paulos et al. assay vary across laboratories, particularly with respect to PCR annealing times and temperatures. Statistical power is limited by small sample sizes, but a few key conditions are plotted in more detail below.

Figure 3: Quantification of measles virus targets (B3, D8, and vaccine strains) using the Wu et al. (Rice) assay. Boxplots show results across different annealing tempatures, faceted by genotype (column) and matrix (row). Result from both wild-type assays (MeV WT) and vaccine-specific assays (MeV VX) are shown.

Higher annealing temperatures (62 degrees C) employed with the Wu et al. assay generally resulted in lower quantification of both B3 and D8 genotypes. Multiple labs that used a 55 degree annealing temperature mis-classified the B3 genotype as vaccine-derived signal. Note that only a small number of participating laboratories (n = 4) employ a vaccine-specific assay.

Figure 4: Quantification of wild-type measles virus targets. Boxplots show results stratified by which wild type probes are used (WT1 only, WT2 only, or both WT1 and WT2).

No obvious differences in performance were apparent between the wild-type probes employed in the Wu et al. assay, but interpretation is limited due to small sample size. Note that all labs using both the WT1 and WT2 probes also used either a 55 or 56 degree annealing temperature, potentially confounding interpretation of probe effects.

Figure 5: Quantification of measles virus target using the Paulos et al. (WWSCAN) assay. Boxplots show results across different annealing tempatures.

Trends with respect to annealing temperature for the Paulos et al. assay are less striking than what was observed for the Wu et al. assay; again, interpretation is severely limited by sample size. See Figure 6 for an analysis of matrix effects.

Matrix Effects

Several samples represented in the testing panel, shown below, differed only in the diluent used when they were prepared: (a) nucleic acids extracted from measles-negative wastewater (“Wastewater”) or (b) 25 mM Tris-HCl (“Buffer”). Note that the wastewater nucleic acids (a) were confirmed negative both from internal testing and in the results from participating laboratories (see Figure 1), where no more than one positive partition was observed.

Figure 6: Quantification of wild-type measles virus targets, faceted by assay (column) and sample composition (row). Notches represent the 95% confidence interval for the median.

Measured concentrations of wild-type measles were substantially higher in wastewater nucleic acid diluent vs. buffer, with significantly different medians independent of the assay used (p < 0.05, Wilcoxon signed-rank). Given that samples were prepared on the day of shipment and experienced only a single freeze-thaw cycle, these results likely reflect relatively rapid measles degradation in the absence of the stabilizing effect provided by additional nucleic acids in the wastewater matrix.

Variability analysis

The data presented in this section were generated as part of separate internal analysis designed to assess the intra-laboratory precision of the PT samples. However, in the course of this testing we observed unexpected results that need to be investigated further.

Preliminary cross-set variability analysis

Multiple additional sample sets were prepared along with the circulated PT samples and tested in-house to generate estimates of variability across the samples shipped to participating labs. These samples were also quantified after 1 week of -80 degrees C storage and re-quantified following an additional freeze-thaw cycle. This testing relied chiefly on the Hummel et al. assay, as it detects both wild-type and vaccine-derived signals, and our routine surveillance assay does not detect the B3 spike-in used for this exercise.

Figure 7: Preliminary internal cross-set variability comparison. The first number in the legend denotes the unique ID of the set tested, with the condition of interest also noted.

The measured cross-set variability was substantially higher than had been observed in previous testing, and furthermore appeared specific to samples containing the D8 genotype. Further investigation found that incidental QIAcuity instrument malfunction had ultimately resulted in delayed thermal cycling of set #1, the highest-quantified sample set shown above.

Effect of time to thermal cycling on positive partitions

A follow-up experiment was performed to directly assess the relationship between total time from priming to imaging completion and wild-type measles quantification using the Hummel et al. assay. Identical template material was used to prepare 8 24-well nanoplates, which were loaded onto two 4-plate QIAcuity instruments. The instruments were operated as normal, with priming performed sequentially on all plates followed by sequential thermal cycling and imaging.

Figure 8: Measles quantification results with Hummel et al. assay vs. duration in hours (time from instrument priming to imaging completion), faceted by QIAcuity instrument (column) and matrix (row). Q1 = 4-plate QIAcuity instrument 1, Q4 = 4-plate QIAcuity instrument 2.

Whereas B3 concentrations remained consistent regardless of when the plate was thermal cycled, for D8, increased time between priming and final imaging (due to plates being thermal cycled in sequence) corresponds to increases in positive partitions.

Furthermore, re-imaging of plates roughly 24 hours after the initial experiment revealed absence of a negative cloud (i.e. aberrant fluorescence in all nanowells). This is highly abnormal, as re-imaging of plates hours to days after initial runs typically does not yield substantially different results than the original run. This has led us to speculate that the increase in positive partitions over time is likely a technical artifact. Importantly, the key factor that distinguishes D8 from B3 in regard to this assay (Hummel et al.) is the 3 mismatches to the reverse primer, but the cause of this phenomenon is currently unknown and has been reported to the instrument manufacturer.

It is important to note that we currently have no evidence of this phenomenon occurring with any of the assays used for routine measles surveillance. However, these results further underscore the importance of sequencing data as vital information to contextualize the performance of wastewater surveillance assays.

Reference material

The wild-type measles spike-ins used for this exercise were:

• Measles Virus, MVs/Tennessee.USA/40.24, Genotype B3
• Measles Virus, MVs/Florida.USA/37.23, Genotype D8